Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add max-depth for check command + global max-depth #791

Merged
merged 13 commits into from
Dec 10, 2021

Conversation

doodlesbykumbi
Copy link
Contributor

@doodlesbykumbi doodlesbykumbi commented Dec 5, 2021

max-depth for the check command limits the depth of the search, a safeguard against particularly expensive queries. This allows users more fine-grain control.

At present the approach in this PR is to

  1. Add max-depth to the check CLI command, and the API (protobufs + HTTP query params)
  2. When max-depth is hit for check, which is compared against the number of recursive (beyond the first) calls to checkOneIndirectionFurther, the search along the given branch halts. Check returning false means a path to a relation is has not been found up to the max-depth.
  3. Adds a global max-depth for all read operations accessible via theserve.read.max-depth key in the config json. This global max-depth takes precedence if the request-level max-depth is less than 1 or if the global max-depth is less than the request-level max depth.

Related issue(s)

closes #703
closes #702

Checklist

  • I have read the contributing guidelines.
  • I have referenced an issue containing the design document if my change
    introduces a new feature.
  • I am following the
    contributing code guidelines.
  • I have read the security policy.
  • I confirm that this pull request does not address a security
    vulnerability. If this pull request addresses a security. vulnerability, I
    confirm that I got green light (please contact
    [email protected]) from the maintainers to push
    the changes.
  • I have added tests that prove my fix is effective or that my feature
    works.
  • I have added or changed the documentation.

Further Comments

Notes for self

  1. Try to run keto locally.
    This was straightforward and so I did so via Github Codespaces. The Makefile shows me how to do most common development activities. I was able to run the cat video example with a local server.
  2. Try to understand the structure of the code.
    • The app is a CLI (based on cobra) that can serve and behave as a client.
    • Serving leveraging gRPC. The version of protoc used is v3.13.0, gleamed from the auto-generated Go files.
    • The "expand" command also makes use of max-depth and so I can crib from that
    • Adding max-depth to "check" command requires similar changes to thosse in the "expand" command: these can be roughly categorised as docs, client, server
    • Running the examples, exploring the code and tests around the "expand" command sheds light on the data model
  3. Implement
    • Make changes to docs and client, to expose maxDepth param and thread it all the way to the "check" handler
    • Make sense of the "check" handler. It uses recursive depth first search?! checkOneIndirectionFurther is the method that is called recursively. NOTE: The search for keto check "*" view videos /cats/1.mp4 wastes diving 2 units of depth while the answer is present at 0 depth.
    • Implement first pass and confirm it through local dev and units
    • Response thoughts
      • Initial thoughts are to decrement maxDepth for each recursive call of checkOneIndirectionFurther and error when maxDepth is spent
      • Alternative approach is to NOT error but instead return the value of "allowed" up to the maxDepth

@CLAassistant
Copy link

CLAassistant commented Dec 5, 2021

CLA assistant check
All committers have signed the CLA.

@doodlesbykumbi doodlesbykumbi force-pushed the check-max-depth branch 2 times, most recently from da97aaa to 600856c Compare December 5, 2021 22:25
@doodlesbykumbi doodlesbykumbi changed the title Add max-depth for check command feat: add max-depth for check command Dec 5, 2021
Copy link
Member

@zepatrik zepatrik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks very good already 👍
I only have a view small comments, but no hurry 😉

Regarding your open questions, I agree with you that returning an error is not the right way to go. Instead, only the branch should be aborted. We could at some point add a boolean to the response that indicates whether not all branches were fully expanded, but I am not sure that there is a good use-case for that.
Except maybe of debugging, but for that you can add a log statement instead of returning an error.

Also, I think the max-depth parameter should not be required, but instead optional. With #703 we will have a configurable fallback that will be used if no value is given.

Thanks again for looking into this.

internal/check/handler.go Outdated Show resolved Hide resolved
proto/install.sh Outdated Show resolved Hide resolved
internal/check/engine.go Show resolved Hide resolved
@doodlesbykumbi doodlesbykumbi changed the title feat: add max-depth for check command feat: add max-depth for check command + global max-depth Dec 6, 2021
@doodlesbykumbi doodlesbykumbi force-pushed the check-max-depth branch 2 times, most recently from ad1b533 to fc40494 Compare December 6, 2021 19:26
@doodlesbykumbi
Copy link
Contributor Author

@zepatrik this should be good for re-review

Copy link
Member

@zepatrik zepatrik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks for all the improvements 👍
There seem to be some not cleaned up test files, I am not sure why sqlite is sometimes keeping them...

  • internal/persistence/sql/TestRelationTupleSubjectTypeCheck
  • internal/persistence/sql/migrations/TestToSingleTableMigrator_HasLegacyTable
  • internal/persistence/sql/migrations/migratest/TestMigrations

internal/check/engine.go Show resolved Hide resolved
internal/check/handler.go Outdated Show resolved Hide resolved
internal/check/handler.go Outdated Show resolved Hide resolved
internal/check/handler.go Outdated Show resolved Hide resolved
internal/check/handler.go Show resolved Hide resolved
internal/expand/engine.go Show resolved Hide resolved
internal/check/engine.go Show resolved Hide resolved
internal/x/max_depth.go Outdated Show resolved Hide resolved
max-depth for the check command limits the depth of the search, a safeguard against particularly expensive queries. This allows users more fine-grain control.
Make max-depth optional for check API. Add global max-depth, for which request-level only takes precedence if it is greater than 0 and less than the global-max depth.
@doodlesbykumbi doodlesbykumbi force-pushed the check-max-depth branch 2 times, most recently from 632eea4 to f9e73e2 Compare December 9, 2021 21:58
Copy link
Member

@zepatrik zepatrik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much, this looks perfect now 👍

@@ -56,6 +64,7 @@ func newCheckCmd() *cobra.Command {

client.RegisterRemoteURLFlags(cmd.Flags())
cmdx.RegisterFormatFlags(cmd.Flags())
cmd.Flags().Int32P(FlagMaxDepth, "d", 0, "Maximum depth of the search tree. If the value is less than 1 or greater than the global max-depth then the global max-depth will be used instead.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this 👍

Comment on lines +26 to +30
type deps struct {
*relationtuple.ManagerWrapper // managerProvider
configProvider
loggerProvider
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, now I get what you mean. I think the previous approach was fine as well, as the manager wrapper type is only a helper for testing, but this is of course the best way to do it 👍

@zepatrik
Copy link
Member

Ah, the only missing thing now is the CLA 😅

@zepatrik zepatrik marked this pull request as ready for review December 10, 2021 10:54
@doodlesbykumbi
Copy link
Contributor Author

@zepatrik CLA done

@zepatrik
Copy link
Member

zepatrik commented Dec 10, 2021

Thank you, CCI is somehow not running... I am looking into it.

No idea why it does not like your fork, pushed now the same branch on this original repo: https://app.circleci.com/pipelines/github/ory/keto/1693/workflows/b840a908-0587-42e5-be87-51ba4a918573

@doodlesbykumbi
Copy link
Contributor Author

doodlesbykumbi commented Dec 10, 2021

@zepatrik It looks like the commit chore: use npm@7 to generate lock-file v2 borked something in linting. You want me to look into it or should I leave that one to you ? Otherwise all else is green

@zepatrik
Copy link
Member

Yeah, for some reason the lock file resulted in a broken state... should be fixed now though

@zepatrik zepatrik merged commit 1e3b63f into ory:master Dec 10, 2021
@zepatrik
Copy link
Member

Thank you very much, greatly appreciate your effort 👍

@doodlesbykumbi doodlesbykumbi deleted the check-max-depth branch December 10, 2021 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add max-depth to global config Add max-depth to Check API
4 participants